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Assessing School Cllnatt in Prevention Program Planning, 

Development, and Evaluation 

Introduction to t hm Sympogtum 

Gallup polls indicate that for each year in the past 
decade school discipline has been the public's number one 
concern about education. During that decade, both the 
National institute of Education and the U.S. Office of 
Juvenile Justice and Delinquency Prevention have undertaken 
programs of research on school disorder and delinquency pre- 
vention. The participants in today's symposium (Cook, 1984; 
Hyman, 1984; D. C. Gottfredson, 1984) have been among the 
most active behavioral scientists conducting this research. 
Their presentations cover some recent applied research 
results in this area, and they comment on the ways the Fed- 
eral government has approached these problems and how it 
might pursue programmatic activities in the future. 

T« Getting Tough All It Takes? 

A recent report by the Cabinet Council on Human Resources 
Working Group on School Violence/Discipline (1984) implies 
that order has deteriorated in the public schools, and it 
fosters notions that (a) school disorder is an acute crisis, 
(b) problems of school disorder can be solved without the 
expenditure of money, and (c) the simple expedient of free- 
ing the hands of school personnel to deal with discipline 
will solve the problems. This report and related adminis- 
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tration views have received widespread attention in the 
media (e.g., Pear, 1984} Weisman, 1984). As X have 
explained elsewhere (G. Gottfredson, 1984) these notions are 
misleading. Rhetoric and admonitions to try harder will not 
help much. Expending the effort to carefully monitor school 
orderliness and carefully implement sound school improvement 
programs will help. Implementing effective school improve- 
ments in the schools that need them will entail the expendi- 
ture of money and the leadership of school personnel and 
others. Expectations that discipline problems in our most 
troubled schools can be solved without resources are unreal- 
istic. In contrast to the "get tough" perspective which 
suggests that the simple and straightforward expedient of 
freeing school officials to manage discipline as they wish — 
-the presenters illustrate that simple expedients are 
unlikely to be effective. Bather than unbridled discretion, 
structures that provide more effective school responses to 
student conduct are needed. 

The presentations cover both primary prevention 
approaches involving environmental change and intervention 
directed at the remediation of difficulties with troublesome 
students. Today's presentations illustrate that the design 
of measurement, intervention, and evaluation procedures have 
developed considerably as a result of the Federal research 
and development initiatives and suggest that additional 
research and development are needed to solve problems of 



adoption and implementation of effective techniques. The 
pursuit of easy "solutions" such as reducing procedural 
hindrances to the increased use of suspension should not be 
allowed to misdirect attention away from research on school 
organization and the development of improved psychological 
technologies. 

Assessing School C] 4mnf » in Prevention Program Planning. 

Development, and Evaluation 

To start the symposium off, I will describe the applica- 
tion of a climate assessment battery in organizational 
development and the evaluation of disciplinary programs in 
schools. This battery is the descendent of research using 
the NIE Safe School Study data, and was developed and field 
tested as part of the evaluation of a school-based delin- 
quency prevention program sponsored by OJJDP. The use of 
this school assessment tool for organizational diagnosis, 
intervention design, and evaluation are illustrated by draw- 
ing on examples from research sponsored by OJJDP and NIE. 

The organizational assessment instruments, known as the 
Effective school Battery (ESB; G. D. Gottfredson, in press 
b) , are based on questionnaires completed by teachers and 
students. I will show profiles of illustrative schools and 
discuss the implications of these assessments for program 
design. Then, I will present selected results from the 
evaluation of OJJDP 1 e school-based delinquency prevention 



program to illustrate the application of the ESB in program 
evaluation. 

Finally, I will contrast the implications of the know- 
ledge gained from the evaluation of the OJJDP program with 
the recently emerged "get tough" approach to school disci- 
pi ine • 

The Effective School Battery (ESB) 

s 

The ESB produces climate profiles to diagnose school 
problems, suggest improvement ideas, and monitor progress. 
The profiles tell how safe a school is, whether morale is 
high or low, whether students and teachers find the school a 
pleasant place to be, and whether there is tension between 
administrators and teachers. In all, the ESB describes the 
school in terms of 34 specific aspects of school climate and 
teacher and student characteristics. The development and 
psychometric properties of the ESB are spelled out in its 
manual (G. D. Gottfredson, in press b) • You should see that 
manual for a summary of the research that led to its devel- 
opment. Today, I will give you a sense for what the results 
of a school assessment using the ESB look like and how they 
can be used. 

Figures 1 through 4 show the profiles for one urban 
school assessed using the ESB. This is a junior high school 
in trouble. It is an urban school located in a working 



class area. Last year one student was shot to death in this 
school, and carrying weapons in the school is commonplace. 
Fights occur all the time. The principal is hanging on 
until retirement. Far from being on top of the school's 
problems, the principal is not sure what the typical daily 
attendance is when asked. (It is low.) 

An incident, most people called it a riot, occurred here 
a few years ago, and central administrators and community 
members alike fear more trouble. Students— and young people 
who are not students — roam throughout the school virtually 
at will. Staff turnover is high. Many teachers put in for 
transfers each year. The principal is not without some 
leadership potential— he is involved in a principal's asso- 
ciation actively working to block the school board's plans 
for a principal evaluation scheme. 

Let's examine this school's climate profiles. The 
teacher psychosocial climate profile summarizes what teach- 
ers say about the school. It shows that botn Safety and 
Morale are very low. The profile suggests that morale is a 
major problem. This interpretation is reinforced by the 
high staff turnover rate mentioned earlier. The low Morale 
score suggests that it will probably be very difficult to 
work with the staff on school improvement programs, but the 
low Safety scale and the generally low elevation of the 
entire profile imply that a school improvement program is 



desperately needed. None of the teacher psychosocial cli- 
mate scales is above average, and three of the more specific 
climate measures are in the low or very low range. 

The pattern seen in this profile suggests inaction rather 
than conflict between faculty and administration. The Plan- 
ning and Action score is low, suggesting that little effort 
is expended on school improvement activities. 

The student psychosocial climate profile summarizes what 
students say about the school. It confirms the interpreta- 
tion that this school is a relatively uncomfortable place. 
Of the two general climate scales, Safety is in the low 
range, and Respect for Students in the moderately low range. 
Like teachers, students see little action: the Planning and 
Action score is low. 

The profile of teacher population characteristics 
describes the teachers in the school. It is marked by very 
low Job Satisfaction, and by low scores on Personal Secur- 
ity, Classroom Orderliness, and Professional Development. 

The profile of student characteristics shows that the 
parents of these students are about as educated as parents 
in the average school. The most striking features of the 
student characteristics are the very low score on Social 
Integration and the very low score on Avoidance of Punish- 
ment. The average student is apparently alienated and is 
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punished often in school. Other evidence confirms that stu- 
dents are often punished — there were 84 disciplinary remo- 
vals (informal suspensions for up to three days) per 100 
students in the year the school was assessed. 

Taken together, the profiles for our illustrative school 
imply that this school has multiple problems, that staff are 
demoralized and students alienated. Anyone trying to help 
this school improve its climate will have to kindle a fire 
under the administration and staff, help set priorities for 
beginning school improvements, and achieve at least some 
small early successes to convince people that something can 
be done to improve matters. 

When these profiles were discussed with the school staff, 
teachers easily named discipline as the number one problem. 
And although the student psychosocial climate profile sug- 
gested that clarity of the rules was at the high end of the 
average range, discussions with teachers and administrators 
revealed that they did not know what the rules were. 

Using fchs ESB to Aaaean School Imp rovement Programs 

In another paper in this symposium, Denise Gottf redson 
(1984) will illustrate how the ESB can be used to assess the 
effects of school improvement interventions. Because she 
will be showing you summary data rather than detailed school 
profiles, it may be useful now to review the way ESB pro- 
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files can be used to describe changes in school climate 
resulting from new programs or other events that can influ- 
ence climate. Figures 5 through 8 depict the climate of one 
school at two points in time. 

This is a junior high school with about 700 students 
located in a mid-sized industrial city. The school was 
desegregated a few years ago and its students are now 67% 
white. The school is probably still settling down after 
this recent court-ordered desegregation and all the city's 
schools are feeling the pinch of tight budgets; the voters 
have turned down referenda for taxes to support education in 
recent years. The principal is a calm and self-assured per- 
son who has captained this school for almost 20 years; the 
vice-principal is a stern disciplinarian who suspends stu- 
dents who look cross-eyed in the hall. The school's suspen- 
sion rate is very high. 

In the spring of 1981 when this school was first 
assessed, the teacher psychosocial climate profile's eleva- 
tion was a bit on the low side of average, suggesting that 
the school was slightly uncomfortable. The Safety scale is 
in the low portion of the average range, suggesting that 
this school has occasional problems in this area, but that 
they are not of great magnitude. 

Several of the more specific climate scales are moder- 
ately low. These include (a) the Planning and Action scale 
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(the principal has been in the school a long time and may 
not be anxious to bring about changes) , (b) the Smooth 
Administration scale (the faculty and counselors say changes 
are needed in the way the vice principal handles disci- 
pline), (c) the Race Relations scale (the school is just 
settling down after the recent court-ordered desegregation), 
and (d) the Student Influence scale (the school has no stu- 
dent council and no one seems concerned about that) • Given 
the talk one hears about the need to get a tax issue 
approved by the voters and the lack of amenities such as 
basketballs, it is interesting that the Resources scale is 
not lower than it is, but then resources like paper and pen- 
cils do not seem to be in short supply, and the school has a 
nice large library. 

Time limitations preclude a detailed examination of the 
details of the remaining profiles (see Gottfredson, in press 
b, for a discussion), but the profile of student population 
characteristics does present some interesting complexities. 
Average Parental Education is moderately high, and several 
indicators of personal integration are near the high end of 
the average range — Positive Peer Association, Educational 
Expectation, Social Integration, Belief in Rules, and Inter- 
personal Competency. But other measures of student charac- 
teristics are moderately low— Attachment to School, Involve- 
ment, Positive Self -Concept, and School Rewards. This 
pattern suggests that something about the school environment 
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is dragging down students 1 liking for school and their 
self-concepts. This interpretation seems to fits The 
School Rewards scale is low, other evidence implies that a 
high proportion of the school's students have been sus- 
pended, and there are very few extracurricular activities 
available. School staff think a large part of the explana- 
tion of the low score on Positive Self-Concept is the way 
disciplinary problems are handled by the vice principal. 
This hypothesis may or may not be correct, but it is worth 
exploration. 

Some important changes occurred in this school between 
1981 and 1983 when the school was again assessed with the 
ESB in the spring. The vice principal left to take another 
job and was replaced by a vice principal with a disciplinary 
philosophy more congenial to the other staff and the central 
administration. The suspension rate plummeted. The voters 
passed a school finance issue during the 1981-82 academic 
year. A graduate student doing an internship in community 
psychology took on the school as an action research project 
and was given an office in the school. The intern had a 
small grant to support the school improvement activities. 
He devoted lots of energy to his school improvement project, 
and he held meetings with faculty and students to plan for 
school improvement. The faculty and administration used the 
ESB assessments conducted in 1981 to develop plans for 
school improvement. A student council was formed, and stu- 




dents launched some fund raising activities that enabled 
them to buy athletic equipment. Dances were held. An in- 
school suspension room was created, and some changes were 
made in reading curricula for students with difficulties. 

The overall elevation of the teacher psychosocial climate 
profile is noticeably higher in 1981. Of the two general 
climate measures, Safety is more than two standard errors of 
measurement higher in 1983 than in 1981, and Morale is high- 
er—but only one and a fifth standard errors higher. Two of 
the more specific climate scales also show evidence of 
improvement. The Smooth Administration scale score has 
increased from the 27th to the 65th percentile, and the 
increase exceeds the standard error of difference by a fac- 
tor of 2.37. Teachers clearly felt more comfortable with 
the administration and that greater cooperation between 
administration and teachers existed in 1983 than they did in 
1981. 

There are no notable changes in the studen. psychosocial 
profile or in the profile of teacher characteristics between 
1981 and 1983. But a number of differences in the measures 
of student population are noteworthy. Social Integration, 
Attachment to School, Belief in Rules, and Positive Self- 
Concept all increased significantly (by simple t-tests) . In 
other words, student alienation has decreased, liking for 
school has gone up, respect for conventional rules is up, 
and students view themselves more positively. 
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This is a portrait of strong positive change in the char- 
acteristics of this school's students. One gets the overall 
picture of a school that is improving. It is not clear 
whether these changes should be attributed solely to the 
school improvement program, especially since there was an 
important change in the school's administration and a school 
revenue issue was passed by the voters. But the pattern of 
results is consistent with this interpretation. 

tlalng the ESB to Plan an d Evaluate Prevention Programa 

A rational approach to planning school improvement to 
reduce disorder would entail the following steps: 1 

1. Diagnosis, use the ESB to assess the school's climate 
and pinpoint the most important problems. Then try to 
determine why the problems exist. If, for example, a cli- 
mate profile implies that school safety is low, search the 
profiles and ask probing questions to find the causes: Are 
the school's rules clear to teachers and students? Is there 
much teacher-administration tension? Is morale so low that 
few people feel they can count on others in the school to 
help them improve discipline? 



1. My colleagues and I have spelled out in greater detail 
elsewhere how to plan, implement, and evaluate school 
improvement programs (G. D. Gottfredson, in press a; G. D. 
Gottfredson, Rickert, D. C. Gottfredson, & Advani, in 
press) • 
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2. Formulate goals and objectives. Be specific about the 

changes in school climate you want to bring about. Base 
your goals and objectives on the diagnosis. If your diagno- 
sis is that the school is unsafe because of a lack of clear 
rules and firm rule enforcement, your goal might be to 
increase safety by making sure the rules are understood by 
all and consistently and fairly enforced. 

3. Examine the research on potential p rogramn. Avoid the 

trap of plunging into a new progrun just because it sounds 
like a good idea. Make sure the programs you put in place 
are clearly and directly aimed at your goals and objectives 
and have been shown to bring them about in other schools. 

A. Identify ohstacl en and resources. When you introduce 
innovation, you are changing the status quo in a school. 
For a while, some things become harder to do while other 
things become easier. You must develop a concrete plan that 
everyone agrees is feasible and which will enable you to 
cope with the obstacles identified. This analysis of obsta- 
cles and resources may reveal that one program is more feas- 
ible than another and help you choose among interventions. 

5. Make a formal plan for school improvement. Once you 
have specified goals and objectives, identified obstacles 
and resources that will influence your program, and selected 
a previously tested innovation, you are ready to make a for- 
mal plan for action. The plan should concretely specify 
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what resources will be used to overcome obstacles, and who 
is responsible for taking what steps by when. Whenever you 
overcome one of your obstacles, you meet a critical bench- 
mark in the implementation of your program. 

6. Specify quality control Bfcandarda. Saying you have a 
program is not the same as having one. You must be specific 
about new policies and procedures. For instance, if you are 
putting new discipline procedures in place, clearly state 
the actions expected in response to specific offenses. Of 
course guidelines may not apply in every instance, so qual- 
ity control standards might specify that disciplinary 
actions will accord with the guidelines, say, 80% of the 
time. 

7. Evaluate, No school improvement program is complete 
without evaluation. Once you have made plans using the for- 
egoing steps, you have a framework for evaluation. Were the 
steps the plan spelled out actually taken? Were the guide- 
lines for the innovation followed? The climate assessment 
using the ESB that was used to formulate the initial diagno- 
sis of the school's problems is a ready-made tool for learn- 
ing if the program is effective. Periodic assessment is a 
straightforward way to chart progress and demonstrate the 
effectiveness of new programs. If school safety was identi- 
fied as a problem, and the diagnosis implied that a lack of 
rule clarity contributed to this problem, then improvements 
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in safety and rule clarity should be visible in the results 
of a new climate assessment with the ESB. 

School Disciplin e Can be Made to Improve 

I want to reassure you that careful planning and the 
implementation of sound programs can lead to safer schools. 
Figure 9 summarises some information about the improvements 
made in a number of schools participating in the School 
Action Effectiveness Study, an evaluation of seventeen 
school-based delinquency prevention projects sponsored by 
the Office of Juvenile Justice and Delinquency Prevention 
(G. D. Gottfredson, 1982; G. D. Gottfredson, D. C. Gottf red- 
son, & Cook, 1983). It compares victimization, classroom 
disorder, and perceptions of safety in 1981 and in 1983, 
after the projects had operated for two years. Although it 
is possible that these improvements in school orderliness 
simply reflect general improvements in the orderliness of 
schools in general over this time period, I regard this as 
unlikely. Our impression is that a major feature distin- 
guishing projects that demonstrated improvements in disci- 
pline in the OJJDP program from those that did not is the 
attention to faithful and thorough implementation of their 
interventions. 
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Figures 1-4: Illustrative Effective School Battery (ESB) Profiles for an Urban Junior High School 
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Figure 2 
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Figure 4 
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Figure 5: A Junior High School Assessed with the ESB in 1981 and 1983 
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Figure 6: A Junior High School Assessed with the ESB in 1981 and 1983 
SCHOOL PSYCHOSOCIAL CLIMATE STUDENT REPORTS 
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Note . The first column under "percentile" 
shows scores for 1 y fl 1 and the 
second column shows scores for 1981. 
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Figure 7; A Junior High School Assessed with the ESB in 1981 and 1983 
SCHOOL POPULATION TEACHER CHARACTERISTICS 
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Note. The first column under "percentile" 
9hows scores for 1981 and the 
second column shows scores for 1983. 



1981 (Response race * SI',) 
1983 (Response rate ■ HI") 



o 

ERLC 



Figure & A Junior High School Assessed with the ESB in 1981 and 1983 
SCHOOL POPULATION STUOENT CHARACTERISTICS 
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SCHOOL REWAROS 21 13 

















Note , The first column under "percentile 11 
shows scores for 1981 and the 
second column shows scores for 1983. 
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BEST COPY AVftlLftBLE 



Figure 9 



CAN SCHOOLS REALLY BE WADE SAFER? 

THE EXPERIENCE OF SCHOOLS IN THE SCHOOL ACTION 
EFFECTIVENESS STUDY: 



STUDENT VICTIMIZATION 

VICTIMIZATION DOWN 
VICTIMIZATION UP 

TEACHER VICTIMIZATION 

VICTIMIZATION DOWN 
VICTIMIZATION UP 

CLASSROOM DISRUPTION 

DECREASED 
INCREASED 

STUDENT PERCEPTIONS OF SAFETY 

SAFETY INCREASED 
SAFETY DECREASED 

TEACHER PERCEPTIONS OF SAFETY 

SAFETY INCREASED 
SAFETY DECREASED 



16 SCHOOLS 
7 SCHOOLS 



27 SCHOOLS'* 
4 SCHOOLS 



25 SCHOOLS" 
7 SCHOOLS 



15 SCHOOLS" 
I SCHOOL 



21 SCHOOLS* 
8 SCHOOLS 



Note Preliminary results from the School Action Effectiveness 
SI nuE of schools differs for the different comparisons 
because comparable data are not available for all measures. 
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